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SIMULATION  AND  TRAINING  FOR  STRESS  ENVIRONMENTS: 
A  META-ANALYTIC  AND  EXPERIMENTAL  EVALUATION 


A.  Research  Objectives: 

The  purpose  of  this  research  was  to  evaluate  the  effectiveness  of  candidate  stress  training 
approaches  for  application  to  the  military  environment,  and  to  identify  specific  guidelines  for 
implementing  these  techniques  to  enhance  the  performance  of  military  personnel. 

B.  Summary: 

The  problem  that  we  have  addressed  in  this  research  project  is  how  to  identify  effective 
training  approaches  to  enhance  the  performance  of  military  personnel  under  high-stress  conditions. 
The  importance  of  "stress  training"  to  the  military  was  identified  as  early  as  World  War  I,  and  a 
large  amount  of  research  has  been  conducted  since  that  time  examining  training  approaches  to  reduce 
the  effects  of  stress  on  performance.  This  research  has  identified  a  number  of  approaches  that  are 
potentially  effective  for  stress  training,  but  the  research  has  often  produced  conflicting  results  that  are 
difficult  to  interpret  at  the  narrative  level. 

Imagine  the  following  scenario.  A  military  program  manager  asks  you  the  following 
questions:  "I  read  that  an  approach  called  stress  inoculation  training  may  be  an  effective  stress 
training  technique,  but  I  also  came  across  some  research  that  showed  no  positive  effects.  Is  this 
training  approach  effective?  Furthermore,  will  it  work  in  a  military  environment?  Is  the  training 
so  complex  to  implement  that  I  need  a  Ph.D-level  trainer,  or  can  I  use  less  experienced  trainers? 

How  many  hours  of  training  are  required  to  enhance  performance?  If  it  takes  40  hours  of  intensive 
training  to  produce  positive  results,  it  won’t  be  of  much  use  in  applied  settings.  Can  you  give  me 
some  answers?" 

There  are  several  different  ways  you  could  attempt  to  answer  these  questions.  First,  you 
could  conduct  an  empirical  study  of  the  effectiveness  of  stress  inoculation  training.  Experimental 
research  is  the  backbone  of  the  scientific  enterprise;  however,  the  results  of  a  single  study  are  rarely 
conclusive  and  often  conflict  with  other  available  data.  Second,  you  could  conduct  a  traditional 
literature  review  and  attempt  to  weigh  the  evidence  from  most  available  studies.  However,  because 
different  studies  are  conducted  in  different  settings,  use  different  outcome  measures,  and  report 
different  study  statistics,  it  is  difficult  if  not  impossible  to  summarize  these  disparate  research 
studies  on  a  narrative  level  in  order  to  draw  specific  conclusions.  A  third  approach  is  to  attempt  to 
integrate  and  summarize  research  results  using  a  meta-analytic  statistical  technique.  This  approach 
uses  a  powerful  statistical  technique  to  summarize  data  from  existing  studies  in  a  systematic  and 
quantitative  manner  in  order  to  provide  specific  answers  to  research  questions. 

The  approach  taken  in  this  project  was  to  conduct  a  meta-analytic  integration  of  the 
research  on  stress  training.  Our  strategy  was  to  examine  those  stress  training  techniques  that  the 
research  literature  suggests  are  potentially  effective,  and  that  are  also  of  interest  to  the  military 
community,  such  as  overlearning,  mental  practice,  and  stress  inoculation  training.  This  approach 
allowed  us  the  opportunity  to  gauge  on  a  quantitative  basis  the  overall  effectiveness  of  alternative 
approaches.  Second,  it  allowed  us  to  identify  factors  that  moderate  the  effectiveness  of  these 
training  approaches,  in  order  to  determine  the  most  effective  means  to  implement  a  specific 
approach.  Finally,  this  strategy  provides  precise  direction  for  further  research  and  application. 


1 


The  Meta-Analvtic  Approach 

It  is  informative  to  provide  a  distinction  between  primary  analysis,  secondary  analysis,  and 
meta-analysis.  Primary  analysis  refers  to  the  original  statistical  analysis  of  data;  for  example,  the 
analysis  of  data  collected  by  a  researcher  examining  the  effects  of  overlearning  on  retention. 
Secondary  analysis  refers  to  analysis  of  data  by  someone  other  than  the  original  investigator.  For 
example,  investigator  B  may  re-analyze  investigator  A's  data  on  the  effects  of  overlearning,  to 
examine  a  particular  variable  of  interest.  Meta-analysis  refers  to  the  analysis  of  the  results  of 
several  independent  studies.  For  example,  if  investigators  A  through  Z  have  conducted  20  studies 
of  the  effects  of  overlearning  on  retention,  a  meta-analysis  would  provide  a  quantitative  summary 
and  integration  of  the  results  of  these  separate  studies. 

Meta-analysis  allows  the  results  of  a  number  of  independent  studies  to  be  analyzed, 
compared,  and  summarized.  For  example,  a  number  of  studies  have  examined  the  effects  of 
mental  rehearsal  or  mental  practice  on  performance.  Some  researchers  have  found  positive  effects 
of  mental  practice,  while  others  have  found  no  significant  effect.  It  is  difficult  to  make  a  confident 
prediction  about  the  effectiveness  of  mental  practice  because  of  the  variability  of  research  findings 
and  the  variety  and  types  of  data  reported  in  these  different  studies.  Glass  (1976)  concurs  that 
"The  accumulated  findings  of  [independent]  studies  should  be  regarded  as  complex  data  points, 
no  more  comprehensible  without  statistical  analysis  than  hundreds  of  data  points  in  a  single  study" 
(p.  352). 

The  meta-analytic  approach  presents  the  opportunity  to  do  two  very  useful  things.  First,  it 
can  provide  a  very  specific  and  precise  summary  of  the  overall  effects  within  a  given  research 
domain  (i.e..  Is  mental  practice  an  eflfective  means  of  enhancing  performance?).  For  example,  by 
analyzing  each  study  that  has  examined  the  effects  of  mental  practice  on  performance,  we  can 
provide  summary  statistics  indicating  the  overall  magnitude  and  significance  of  this  effect. 

Second,  this  strategy  allows  us  to  test  specific  relationships  and  theoretical  assumptions  which 
would  be  exorbitantly  expensive,  or  practically  impossible,  to  examine  at  a  primary  level  of 
analysis.  For  example,  we  are  able  to  examine  factors,  such  as  amount  of  training  or  type  of  task, 
that  may  moderate  the  effects  of  mental  practice  on  performance.  In  this  manner,  meta-analytic 
results  can  provide  specific  guidelines  to  the  military  training  designer  on  how  to  most  effectively 
implement  training  (i.e.;  How  many  hours  of  mental  practice  are  required  to  impact  performance? 
When  should  refi’esher  training  be  provided?)  By  integrating  the  results  of  studies  of  mental 
practice,  overlearning,  and  other  training  approaches  through  the  use  of  meta-analytic  techniques, 
we  can  not  only  provide  a  very  accurate  estimate  of  the  effectiveness  of  each  of  these  training 
approaches,  but  also  identify  how  they  can  be  most  effectively  implemented. 

C.  Research  Results: 

Research  was  conducted  to  examine  overlearning,  mental  practice,  stress  inoculation 
training,  cohesiveness,  team  building  and  relaxation  training.  Results  are  briefly  discussed  below. 

The  efficacy  of  overlearning  on  enhancing  performance  has  been  acknowledged  by 
researchers  within  the  training  community  for  years,  and  this  approach  has  been  implemented  in 
several  military  applications.  In  spite  of  this  general  consensus,  there  is  little  summary  data  on  the 
effectiveness  of  this  technique,  nor  are  there  specific  guidelines  on  how  to  implement  overlearning 
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(i.e.,  what  degree  of  overlearning  is  required  to  enhance  performance,  how  long  the  beneficial 
effects  of  overlearning  are  retained  over  time,  etc.)  Results  of  our  analysis  indicated  that 
overlearning  is  an  effective  means  for  enhancing  retention  for  both  physical  and  cognitive  tasks. 
Furthermore,  the  results  of  this  analysis  produced  practical  guidelines  for  implementing 
overlearning  in  operational  settings.  For  example,  results  revealed  that  (a)  the  increase  in  retention 
due  to  overlearning  dissipates  after  approximately  5-6  weeks,  therefore  maintaining  optimal 
performance  requires  that  additional  training  take  place  on  a  relatively  short-term  (3-4  week) 
schedule,  (b)  this  decay  is  less  evident  for  physical  tasks,  thus  there  may  be  considerable  practical 
value  in  ensuring  active,  physical  practice  of  skills  during  training,  and  (c)  the  increased  initial 
costs  associated  with  overlearning  may  be  partially  offset  by  lowered  costs  for  subsequent 
retraining  or  refresher  training. 

Mental  practice  refers  to  the  symbolic  rehearsal  of  an  activity  in  the  absence  of  overt 
physical  performance.  A  recent  National  Research  Council  report  on  training  techniques 
concluded  that  "the  available  research  on  mental  practice  is  consistent  enough  to  support  a 
recommendation  for  the  Army  to  conduct  evaluation  studies  on  operational  military  tasks" 
(Druckman  &  Swets,  1988,  p.  70).  However,  a  number  of  questions  arise  regarding  this 
technique:  Some  argue  that  the  effectiveness  of  mental  practice  is  limited  to  cognitive  tasks,  some 
claim  that  previous  experience  with  the  task  is  a  prerequisite  for  the  effective  use  of  mental 
practice,  and  others  argue  that  "too  much"  mental  practice  can  have  negative  effects.  Results  of 
our  analysis  revealed  that  mental  practice  is  an  effective  means  for  enhancing  performance,  and 
that  the  effectiveness  of  mental  practice  is  moderated  by  the  type  of  task,  the  retention  interval 
between  practice  and  performance,  and  the  length  or  duration  of  the  mental  practice  intervention. 

Unit  cohesiveness  has  been  identified  by  many  as  the  single  most  important  variable  in 
reducing  the  impact  of  stress  in  combat.  However,  some  studies  call  into  question  the  relationship 
between  cohesiveness  and  performance.  In  fact,  a  recent  National  Research  Council  review 
concluded  that  of  the  topics  studied,  "none  shows  a  larger  discrepancy  between  what  we  think  we 
know  and  the  existing  evidence  than  that  of  cohesion"  (Druckman  &  Swets,  1988,  p.  133). 

Results  of  our  analysis  indicated  that  the  overall  effect  of  cohesiveness  on  performance  was  highly 
significant  and  of  small  magnitude.  The  effect  of  cohesiveness  on  performance  was  moderated  by 
the  type  of  group  (real-world  versus  artificial  groups)  and  by  the  dimension  of  cohesiveness  that 
was  operationalized.  Furthermore,  a  meta-analytic  cross-lagged  panel  correlation  analysis 
suggested  that  the  direction  of  effect  is  stronger  from  performance  to  cohesiveness  than  from 
cohesiveness  to  performance. 

Stress  inoculation  training  (SIT)  is  a  cognitive-behavioral  approach  to  stress  management 
that  has  shown  considerable  promise;  however,  a  number  of  questions  arise  regarding  the 
application  of  this  approach  to  an  applied  training  environment.  For  example,  stress  inoculation 
training  has  been  implemented  primarily  as  a  clinical  approach,  in  a  laboratoiy  setting,  and  often 
administered  on  a  one-to-one  basis  by  an  experienced  trainer.  Therefore,  the  applicability  of  this 
approach  to  a  more  applied  training  environment,  in  which  training  is  often  administered  on  a 
group  basis  by  a  non-Ph.D.  level  trainer,  is  an  important  issue.  A  meta-analysis  of  the  literature 
on  stress  inoculation  training  was  conducted  to  determine  the  effectiveness  of  SIT  and  to  identify 
conditions  that  may  moderate  the  effectiveness  of  this  intervention.  Results  indicated  that  SIT  is 
an  effective  means  for  reducing  state  anxiety,  reducing  skill-specific  anxiety,  and  enhancing 
performance  under  stress.  Furthermore,  the  examination  of  moderators  such  as  the  experience  of 
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the  trainer,  type  of  setting  in  which  SIT  is  implemented,  and  type  of  trainee  population  revealed 
no  significant  limitations  on  the  application  of  SIT  to  military  training  environments. 

Progressive  relaxation  training  is  one  of  the  most  commonly  used  stress  reduction 
techniques  in  the  research  literature.  However,  empirical  studies  of  training  effectiveness  are 
equivocal.  A  meta-analysis  of  the  literature  on  relaxation  training  was  conducted  to  determine  the 
overall  effects  of  relaxation  training  and  identify  conditions  under  which  relaxation  training  is 
most  effective.  Results  indicated  that  relaxation  training  has  a  positive  and  significant  effect  on 
reducing  subjective  anxiety.  Moreover,  moderators  such  as  the  type  of  subject  population,  mode 
of  presentation,  and  size  of  the  training  group  revealed  that  relaxation  training  is  equally  effective 
in  a  variety  of  different  environments  and  under  different  conditions.  However,  there  is  a  lack  of 
data  on  the  effects  of  relaxation  training  on  performance.  Therefore,  although  relaxation  training 
may  make  an  individual  feel  less  stressed,  trainers  should  be  cautioned  that  the  effects  of  this 
approach  on  task  performance  are  unclear. 

Team-building  is  one  of  the  most  frequently  used  organization  development  interventions. 
However,  some  research  reviews  have  concluded  that  there  is  no  clear-cut  evidence  to  suggest 
that  team  building  can  improve  the  performance  of  individuals  or  teams.  The  purpose  of  this 
analysis  was  to  integrate  the  team  building  literature  and  examine  the  relationship  between  team 
building  and  performance.  The  results  demonstrated  that  the  team  building  intervention  improved 
objective  performance  of  the  team  and  enhanced  team  members'  affective  reactions  to  the  team. 
However,  the  effects  of  team  building  were  moderated  by  a  number  of  important  variables.  For 
example,  team  building  was  more  effective  in  enhancing  team  performance  for  groups  of  smaller 
size,  it  was  more  effective  for  managerial  than  for  blue  collar  work  groups,  and  it  was  more 
effective  for  enhancing  performance  of  newly-formed  work  groups  than  for  intact  work  groups. 

D.  Contributions  of  this  Research: 

One  unique  benefit  of  this  approach  is  the  goal  of  aggregating  and  cumulating  the  results  of 
the  large  body  of  research  on  stress  and  training  effectiveness  that  has  been  to  a  large  degree 
supported  and  funded  by  the  military  services.  This  approach,  in  effect,  "leverages"  the 
investment  in  this  prior  research  by  comparing,  integrating,  and  interpreting  this  research  to 
summarize  research  results  and  determine  high-priority  directions  for  further  research  and 
application.  Furthermore,  the  results  of  this  research  provide  very  precise  guidelines  for 
implementing  stress  training  to  enhance  the  performance  of  military  personnel. 

E.  Publications  and  Presentations: 

Publications: 

Driskell,  J.  E.,  &  Salas,  E.  (Eds.)  (in  press).  Stress  and  human  performance.  Hillsdale,  NJ: 
Erlbaum. 

Mullen,  B.,  &  Copper,  C.  (1994).  The  relation  between  group  cohesiveness  and  performance:  An 
integration.  Psychological  Bulletin.  115,  210-227. 

Driskell,  J.  E.,  Copper,  C.,  &  Moran,  A.  (1994).  Does  mental  practice  enhance  performance? 
Journal  of  Applied  Psychology.  79,  481-492. 


4 


Mullen,  B.,  Anthony,  T.,  Salas,  E.,  &  Driskell,  J.  E.  (1994).  Group  cohesiveness  and  quality  of 
decision  making:  An  integration  of  tests  of  the  groupthink  hypothesis.  Small  Group  Research. 
25,  189-204. 

Driskell,  J.  E.,  Willis,  R.,  &  Copper,  C.  (1992).  Effect  of  overlearning  on  retention.  Journal  of 
Applied  Psychology.  77,  615-622. 

Driskell,  J.  E.,  &  Salas,  E.  (1992).  Military  psychology.  In  T.  N.  Dupuy  (Ed.),  International 
military  and  defense  encyclopedia.  Washington,  DC:  Pergamon. 

Saunders,  T.  Driskell,  J.  E.,  Hall,  J.,  &  Salas,  E.  (1995).  The  effect  of  stress  inoculation  training 
on  anxiety  and  performance.  Manuscript  under  review. 

Driskell,  J.  E.,  and  Miller,  L.  A.  (1994).  Stress  and  relaxation  training:  A  meta-analvtic  review. 
Manuscript  under  review. 

Presentations: 

Driskell,  J.  E.,  &  Salas,  E.  (1995).  A  meta-analvtic  integration  of  stress  training  approaches. 
Paper  presented  at  the  annual  meeting  of  the  American  Psychological  Association,  New  York. 

Driskell,  J.  E.  (1994).  The  application  of  behavioral  science  to  real-world  problems.  Symposium 
presented  at  the  American  Association  for  the  Advancement  of  Science,  San  Francisco. 

Miller,  L.  A.,  &  Driskell,  J.  E.  (1994).  The  effect  of  relaxation  training  on  anxiety.  Paper  to  be 
presented  at  the  annual  meeting  of  the  American  Psychological  Association,  Los  Angeles. 

Driskell,  J.  E.,  Copper,  C.,  &  Moran,  A.  (1993).  Does  mental  practice  enhance  performance? 
Paper  presented  at  the  annual  meeting  of  the  American  Psychological  Association,  Toronto. 

Saunders,  T.,  Driskell,  J.  E.,  Hall,  J.,  &  Salas,  E.  (1993).  The  effect  of  stress  inoculation  training 
on  anxiety  and  performance.  Paper  presented  at  the  annual  meeting  of  the  American 
Psychological  Association,  Toronto. 

Mullen,  B.,  Anthony,  T.,  Salas,  E.,  &  Driskell,  J.  E.  (1993).  Group  cohesiveness  and  quality  of 
decision  making.  Paper  presented  at  the  annual  meeting  of  the  Society  for  Industrial  and 
Organizational  Psychology,  Toronto. 

Kaufmann,  B.,  &  Mullen,  B.  (1993).  The  presence  of  others  and  arousal:  An  integration.  Paper 
presented  at  the  annual  meeting  of  the  Eastern  Psychological  Association,  Arlington,  VA. 

Driskell,  J.  E.,  Mullen,  B  .,  Willis,  R.,  &  Copper,  C.  (1991).  The  effect  of  overlearning  on 

retention.  Paper  presented  at  the  annual  meeting  of  the  American  Psychological  Association, 
San  Francisco^  - 


5 


